Real-time online-learning object recommendation engine

ABSTRACT

In one embodiment, a method includes receiving a request from a first user for a content page; in response to the request, identifying at least one content page, wherein the content page is associated with a page identifier; identifying a plurality of content items based at least in part on a plurality of content features associated with the content page; ranking the plurality of content items based at least in part on a plurality of user features associated with the first user; and delivering to the first user, with the requested content page, one or more of the plurality of content items as recommendations to the first user based on the ranking of the content items.

PRIORITY

This application is a continuation under 35 U.S.C. §120 of U.S. patentapplication Ser. No. 13/313984, filed 7 Dec. 2011.

TECHNICAL FIELD

This disclosure relates generally to the field of social networking.

BACKGROUND

Computer users are able to access and share vast amounts of informationthrough various local and wide area computer networks includingproprietary networks as well as public networks such as the Internet.Typically, a web browser installed on a user's computing devicefacilitates access to and interaction with information located atvarious network servers identified by, for example, associated uniformresource locators (URLs). Conventional approaches to enable sharing ofuser-generated content include various information sharing technologiesor platforms such as social networking websites. Such websites mayinclude, be linked with, or provide a platform for applications enablingusers to view “profile” pages created or customized by other users wherevisibility and interaction with such profiles by other users is governedby some characteristic set of rules. By way of example, a user profilemay include such user-declared information as contact information,background information, job/career information, as well as interests.

A traditional social network is a social structure made of individuals,groups, entities, or organizations generally referred to as “nodes,”which are tied (connected) by one or more specific types ofinterdependency. Social network (graph) analysis views socialrelationships in terms of network theory consisting of nodes and edges.Nodes are the individual actors within the networks, and edges are therelationships between the actors. The resulting graph-based structuresare often very complex. There can be many kinds of edges between nodes.In its simplest form, a social network, or social graph, is a map of allof the relevant edges between all the nodes being studied.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example computer network environment of an examplesocial network environment.

FIG. 2 illustrates an example profile page including recommendedobjects.

FIG. 2A illustrates another example profile page including recommendedobjects.

FIG. 3 illustrates an example system architecture for an objectrecommendation engine in accordance with one embodiment.

FIG. 4 illustrates an example method for generating recommended objectsin response to a request.

FIG. 5 illustrates an example method for updating a model for an onlinelearner.

FIG. 6 illustrates an example network environment.

FIG. 7 illustrates an example computer system architecture.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Particular embodiments relate to a social network environment thatincludes an infrastructure or platform (hereinafter infrastructure andplatform may be used interchangeably) enabling an integrated socialnetwork environment. In this disclosure, the social network environmentmay be described in terms of a social graph including social graphinformation. In particular embodiments, one or more computing systems ofthe social network environment implementing the social networkenvironment include, store, or have access to a data structure thatincludes social graph information for use in implementing the socialnetwork environment described herein. The social network utilizes asocial graph that includes nodes representing users and concepts in thesocial network environment as well as edges that define or representconnections between such nodes. In particular embodiments, any objectwithin the social networking system or node within the social graph maybe ranked or scored via a recommendation engine that predicts theexpected value of the object or node to the user. Particular embodimentsrelate to presenting objects or nodes to a particular user that maximizethe expected value of the entire system.

In particular embodiments, the social graph information includes a firstset of user nodes that each correspond to a respective user, and asecond set of concept nodes that each correspond to a respectiveconcept. As used herein, a “user” may be an individual (human user), anentity (e.g., an enterprise, business, or third party application), or agroup (e.g., of individuals or entities) that interacts or communicateswith or over such a social network environment. As used herein, a“concept” may refer to virtually anything that a user may declare orotherwise demonstrate an interest in, a like towards, or a relationshipwith, such as, by way of example, a sport, a sports team, a genre ofmusic, a musical composer, a hobby, a business (enterprise), an entity,a group, a third party application, a celebrity, a person who is not aregistered user, etc. In particular embodiments, each node has,represents, or is represented by, a corresponding web page (“profilepage”) hosted or accessible in the social network environment. Inparticular embodiments, the social graph information further includes aplurality of edges that each define or represent a connection between acorresponding pair of nodes in the social graph.

By way of example, a user node may have a corresponding user profilepage in which the corresponding user can add content, make declarations,and otherwise express him or herself, while a concept node may have acorresponding concept profile page (“hub”) in which a plurality of userscan add content, make declarations, and express themselves, particularlyin relation to the concept. For example a place, business, or brand maybe represented on the social graph by a concept node and itscorresponding hub page, and users may make connections to the conceptnode via an expression of affinity, such as a “like”, commenting on thehub page, an expression that the user was physically at the location by“checking in,” and the like. Such connections may be represented, inparticular embodiments, by edges connecting the user node to the conceptnode. As another example, events may be represented by a node on asocial graph, and users may broadcast their intent to attend the eventthrough an RSVP edge.

In particular embodiments, any piece of content on the social networkingsystem may be considered an object or a node in the social graph. Forexample, the individual status updates a member writes, or the photos inhis or her albums, may each be an object in the social graph. Inparticular embodiments, objects in the social graph may be “open”,meaning third-party developers may utilize an application programminginterface to generate custom object types, as well as custom actionsthat users may perform on the objects, in the social graph.

In particular embodiments, the social networking system may make objectrecommendations to users browsing the social networking web site.Particular embodiments relate to utilizing a real-time online learningobject recommendation engine to rank particular content objects based ontheir expected interest level to a particular user, and render the mostinteresting objects to users. In particular embodiments, therecommendation engine possesses an online learning component that logswhether a user converts, or acts upon, a particular content object afterthe initial display, or impression, and updates the recommendationengine substantially in real time with the conversion or lack thereof.

Various portions of such a social networking platform may be implementedvia a hardware architecture or software framework that enables varioussoftware components or processes to implement particular embodiments, asis described in more detail, by way of example and not by way oflimitation, below. The platform may include one or more hardware orsoftware components, one or more of which may be located or embodied inone or more consolidated or distributed computing systems. Additionally,as used herein, “or” may imply “and” as well as “or;” that is, “or” doesnot necessarily preclude “and,” unless explicitly stated or implicitlyimplied. As just described, in various example embodiments, one or moredescribed web pages or web applications are associated with a socialnetwork environment or social networking service.

Particular embodiments may operate in, or in conjunction with, a widearea network environment, such as the Internet, including multiplenetwork addressable systems. FIG. 1 illustrates an example networkenvironment, in which various example embodiments may operate. Internet120 generally represents one or more interconnected networks, over whichvarious systems and hosts described herein may communicate. Internet 120may include packet-based wide area networks (such as the Internet),private networks, wireless networks, satellite networks, cellularnetworks, paging networks, and the like. As FIG. 1 illustrates,particular embodiments may operate in conjunction with a networkenvironment comprising social network system 100 and client devices 113a-c, as well as, in some embodiments, one or more third party webapplication servers 140. Client devices 113 a-c, web servers 40, andthird-party websites 140 may be operably connected to the networkenvironment and Internet 120 via a network service provider, a wirelesscarrier, a set of routers or networking switches, or any other suitablemeans.

Each client device 113 may generally be a computer, computing system, orcomputing device including functionality for communicating (e.g.,remotely) over a computer network. Client device 113 in particular maybe a desktop computer 113 c, laptop computer, personal digital assistant(PDA), tablet PC 113 b, in- or out-of-car navigation system, smart phone113 a or other cellular or mobile device, or mobile gaming device, amongother suitable computing devices. Client device 113 may execute one ormore client applications, such as a web browser (e.g., MICROSOFT WINDOWSINTERNET EXPLORER, MOZILLA FIREFOX, APPLE SAFARI, GOOGLE CHROME, ANDOPERA, etc.),to access and view content over a Internet 120. Inparticular implementations, the client applications allow a user ofclient device 113 to enter addresses of specific network resources to beretrieved, such as resources hosted by social networking system 100.These addresses can be Uniform Resource Locators (URLs). In addition,once a page or other resource has been retrieved, the clientapplications may provide access to other pages or records when the user“clicks” on hyperlinks to other resources. By way of example, suchhyperlinks may be located within the web pages and provide an automatedway for the user to enter the URL of another page and to retrieve thatpage.

More particularly, when a user at a client device 113 desires to view aparticular web page (hereinafter also referred to as a target structureddocument) hosted by social networking system 100, and made available inconjunction with social networking system 100, the user's web browser,or other client-side structured document rendering engine or suitableclient application, formulates and transmits a request to web servers101 of social networking system 100. The request generally includes aURL or other document identifier as well as metadata or otherinformation. By way of example, the request may include informationidentifying the user, such as a user ID, as well as informationidentifying or characterizing the web browser or operating systemrunning on the user's client computing device 113. The request may alsoinclude location information identifying a geographic location of theuser's client device or a logical network location of the user's clientdevice, as well as timestamp identifying when the request wastransmitted.

In an example implementation, when a request for a web page orstructured document hosted by social networking system 100 is receivedby web servers 101 of social networking system 100, one or morepage-generating processes executing within the social networking system100 typically generates a base web page in the form of a Hyper TextMarkup Language (HTML), Extensible Markup Language (XML), or other webbrowser-supported structured document. The generated structured documentis then transmitted in a response, which may comprise one or moreportions or partial responses, to the requesting client 113 via aHypertext Transfer Protocol (HTTP) or other suitable connection forrendering by the web browser at the client device 113. The structureddocument may include one or more resources (e.g. JavaScript scripts,code segments, or resources, Cascading Style Sheet (CSS) code segmentsor resources, image data or resources, video data or resources, etc.),or references to such resources, embedded within the transmitteddocument. By way of example, a resource embedded in an HTML document maygenerally be included or specified within a script element, imageelement, or object element, among others, depending on the type ofresource. The element referencing or specifying the resource may includea source attribute (e.g., src) identifying a location of the resource,which may be within a server or data store within social networkingsystem 100 or at one or more external locations, to the client device113 requesting the web page. Typically, upon receipt of the response,the web browser or other client document rendering application runningat the client device 113 then constructs a document object model (DOM)representation of the received structured document and requests theresource(s) (which may be at one or more other external locations)embedded in the document.

In an example implementation, when a registered user of socialnetworking system 100 first requests a web page from social networkingsystem 100 in a given user session, the response transmitted to theuser's client device 113 from web servers 101 of social networkingsystem 100 may include a structured document generated by thepage-generating process for rendering a login page at the client device.The user may then enter his or her user login credentials (e.g., user IDand password), which are then transmitted from the user's client device113 to social networking system 100. Upon successful authentication ofthe user, social networking system 100 may then transmit a response tothe user's web browser at the user's client device 113 that includes astructured document generated by the page-generating process forrendering a user homepage or user profile page at the user's clientdevice.

In one example embodiment, social networking system 100 comprisescomputing systems that allow users at client devices 113 to communicateor otherwise interact with each other and access content, such as userprofiles, as described herein. Social networking system 100 is a networkaddressable system that, in various example embodiments, comprises oneor more physical servers as well as one or more data stores.. The one ormore physical servers are operably connected to Internet 120 via, by wayof example, a set of routers or networking switches. In an exampleembodiment, the functionality hosted by the one or more physical serversmay include web or HTTP servers, FTP servers, as well as, withoutlimitation, web pages and applications implemented using Common GatewayInterface (CGI) script, PHP Hyper-text Preprocessor (PHP), Active ServerPages (ASP), Hyper Text Markup Language (HTML), Extensible MarkupLanguage (XML), Java, JavaScript, Asynchronous JavaScript and XML(AJAX), and the like.

The physical servers may host functionality directed to the operationsof social networking system 100. By way of example, social networkingsystem 100 may host a website that allows one or more users, at one ormore client devices 113, to view and post information, as well ascommunicate with one another via the website. Hereinafter, the physicalservers may be referred to as web server 101, although, as justdescribed, the servers may include numerous servers hosting, forexample, social networking system 100, as well as other contentdistribution servers, data stores, or databases.

The data store may comprise object store 102, which may store contentand data relating to, and enabling, operation of the social networkenvironment as digital data objects including content objects. A dataobject, in a particular implementation, is an item of digitalinformation typically stored or embodied in a data file, database, orrecord. Content objects may take many forms, including: text (e.g.,ASCII, SGML, HTML), images (e.g., jpeg, tif and gif), graphics(vector-based or bitmap), audio, video (e.g., mpeg), or othermultimedia, and combinations thereof. Content object data may alsoinclude executable code objects (e.g., games executable within a browserwindow or frame), podcasts, etc. Logically, object store 102 correspondsto one or more of a variety of separate or integrated databases, such asrelational databases and object-oriented databases, that maintaininformation as an integrated collection of logically related records orfiles stored on one or more physical systems.

In particular embodiments, object store 102 may include a number ofdatabases. For example object store 102 may include user database 103containing information pertaining to every single user of socialnetworking system 100. In particular embodiments, object store 102 mayinclude places database 104, containing information pertaining tophysical locations or businesses represented by concept nodes in socialnetworking system 100. In particular embodiments, users or other nodeadministrators may create event pages for particular real-world eventsusers may attend. Information regarding these events may be stored inevents database 105. In particular embodiments, media objects, such asphotos, albums, videos, and the like may be stored in media database106. In particular embodiments, users may share links to other contenthosted on third party websites 140 or external application servers. Inparticular embodiments, each link may be represented by a node on thesocial graph. In particular embodiments, shared links may be stored asobjects within share db 107, or alternatively story database 110. Inparticular embodiments, each concept node may be represented by a hubpage, and both the nodes and pages may be stored in pages database 106.In particular embodiments, individual advertisements may be treated asobjects within social networking system 100, and advertisements may bestored in ad db 109. In particular embodiments, individual newsfeedstories, status updates, check-ins, and the like may be stored in storydatabase 110. Finally custom objects defined by third-parties may bestored in other object store 111. Individual databases 103-111 may belocated on one or more physical servers. In particular embodiments,individual databases 103-111 may be located in a single server. Thisdisclosure contemplates any suitable means of storing objects of socialnetworking system 100 in any physical configuration.

Structurally, object store 102 may generally include one or more of alarge class of data storage and management systems. In particularembodiments, object store 102 may be implemented by any suitablephysical system(s) including components, such as one or more databaseservers, mass storage media, media library systems, storage areanetworks, data storage clouds, and the like. In one example embodiment,object store 102 includes one or more servers, databases (e.g., MySQL),and/or data warehouses.

Object store 102 may include data associated with different socialnetworking system 100 users, client devices 113, web servers 101 as wellas, in particular embodiments, data associated with various concepts. Asdescribed above, particular embodiments relate to a social networkingsystem 100 that includes a platform enabling an integrated socialnetwork environment. In the following example embodiments, the socialnetwork environment may be described or implemented in terms of a socialgraph including social graph information. In particular embodiments,object store 102 includes a the social graph database in which thesocial graph information for use in implementing the social networkenvironment described herein is stored. In particular embodiments, thesocial graph information stored by social networking system 100 inobject store 102, and particularly in the social graph database,includes a plurality of nodes and a plurality of edges that defineconnections between corresponding nodes. In particular embodiments, thenodes or edges themselves are data objects that include the identifiers,attributes, and information (including the information for theircorresponding profile pages) for their corresponding users or concepts(as described below), some of which is actually rendered oncorresponding profile or other pages. The nodes may also includepointers or references to other objects, data structures, or resourcesfor use in rendering content in conjunction with the rendering of theprofile pages corresponding to the respective nodes.

Social networking system 100 may include object recommendation engine112. Object recommendation engine may determine, for any given user andany given object in object store 102, the expected probability that theparticular user may find the object engaging. In particular embodiments,object recommendation engine 112 utilizes a probabilistic model tocalculate an expected value for the user. Object recommendation engine112 is described further with reference to FIG. 3.

FIGS. 2 and 2A illustrate example profile pages including objectsrecommended to a particular user by social networking system 100. InFIG. 2, profile page 201 includes multiple elements that are generatedby social networking system 100 and delivered to a user's client device113 a-c via web servers 101. Profile page 201 may include, for example,the user's profile picture 202, basic information 203, and a newsfeed204 of the recent or most interesting activities of the user'sconnections. In particular embodiments, profile page 201 includes aright-hand column 205 that may include dynamically-generated anduser-specific objects for display to the user. For example, right-handcolumn 205 may include event invitations 205 a, a user suggestion object(“people you may know”) 205 b, sponsored stories 205 c, andadvertisements 205 d. In particular embodiments, each of these elementsmay be a specific object within social networking system 100 that may bescored and ranked specifically for the user such that the mostinteresting content is delivered on the right-hand column. For example,when a user is viewing his or her profile page, the user may be mostinterested in connecting with new users, rather than, for example,browsing his or her existing connections' photo albums. In particularembodiments, social networking system takes into account the user'scurrent page context in determining the type of objects to recommend tothe user. Even having filtered the potential objects, for didacticpurposes in FIG. 2, to new users, social networking system 100 mustdetermine which users the viewing user will find the most interesting,or, in other words, which users the viewing user is most likely to clickon. In particular embodiments, social networking system utilizes aheuristic model based on the viewing user's previous viewing history tocalculate an expected value for each user object candidate to bedisplayed to the viewing user.

FIG. 2A illustrates another example page view having objectrecommendations 206 and 207. In this example, the viewing user iscurrently on a specific photo page for one of his first-degreeconnections. As previously disclosed, social networking system 100 mayfilter the list of candidate objects based on the currently requestedpage context. In the example of FIG. 2A, a plurality of individuals aretagged within the viewed photo, including the user “Dong Siying” 208.Because the viewing user has requested a specific photo of one of hisfriends, social networking system 100 determines the user will be moreinterested, and therefore more likely to click, photo albums owned byone of the members of the social networking system that is tagged in theviewed photo, and therefore recommends photo albums 206 and 207, ownedby “Dong SiYing”, to the viewing user in the right-hand column of thecurrently viewed page. Although FIGS. 2 and 2A display recommendedobjects on the right-hand column, this disclosure contemplates thedisplay of recommended objects in any portion of the requesting user'sdisplay.

FIG. 3 illustrates the system architecture of recommendation engine 112.Recommendation engine 112 is coupled via data connections or networklinks to web servers 101, recommendation candidate index 301,recommendation content feature provider 302, user feature provider 303,and online learner 304. The various components depicted in FIG. 3 may behoused in separate servers, in the same server, or each individuallydistributed across multiple servers. In particular embodiments, one ormore of the components, for example recommendation content featureprovider 302 and user feature provider 303, may be logically orphysically collocated with object store 102. FIG. 3 provides a logicalsystem architecture for recommendation engine 112; this disclosurecontemplates any physical configuration of servers, applications, ornetworks implementing recommendation engine 112. FIG. 3 also depictsdata exchanges between recommendation engine 112 and the variouscomponents of social networking system 100.

When a user at a client device such as 113 a-c requests a page from webservers 101, for example, his or her profile page, the web serverstransmit data to recommendation engine 112 to request recommendedobjects to display to the user. In particular embodiments, web servers101 forward the requesting user's social networking ID and the page ID.The page ID allows recommendation engine 112 to determine the context ofthe page, for example, whether the user is viewing his or her ownprofile page, another user's profile page, an event page, a photo page,a hub page, his or her inbox, a specific message thread, or the like.Based on the user ID and page ID, recommendation engine 112, oralternatively recommendation candidate index 301, may determine thepeople on the page and, in particular embodiments, the people who arethe user's friends on the page. In the example of FIG. 2A, when theviewing user requests to view a page including a photo, recommendationcandidate index 301 may determine, based on the page ID, the people onthe page, and by querying the social graph with the viewing user's IDand the people on the page, may determine the people in the photographwho are also friends with the user. In particular embodiments,recommendation candidate index may filter the objects in object store102 by this information.

Thus, in particular embodiments, the universe of recommendable objectsis filtered by recommendation candidate index 301 based on severalfactors. First, the objects may be filtered based on page context. Forexample, if a user is viewing his or her home page or profile,recommendation candidate index may limit the objects to be recommendedto other users the viewing user may know. As another example, if theuser is viewing a particular other user's profile, the objects may befiltered down to photos of the particular user, or previous statusmessages by the particular user. Second, the objects may be filteredbased on the people contained within the requested page. For example, ifthe user is viewing a conversation thread with another particular user,recommendation candidate index 301 may limit the objects to berecommended to photos and status updates from that particular user. Asanother example, if the user is viewing an event page, recommendationcandidate index 301 may filter the recommendable objects to other eventsthat the user's friends are attending, or, even more specifically, otherevents that the user's friends, who are attending the viewed event page,are also attending.

In particular embodiments, candidate recommendation index 301 performsall the cross-referencing of the user ID, page ID, and people containedin the page represented by the page ID, to filter the objects to berecommended to the user. However, given the number of page requests toweb servers 101 in a social networking system that may have upwards of800 million users, particular embodiments contemplate shifting some ofthe load to web servers 101 to streamline the process. In particularembodiments, web servers 101 transmit a user ID, page ID, and a list ofusers contained in the page to recommendation engine 112. Web servers101 may easily parse the requested page to be delivered to the user forthe other users contained in the page, thereby obviating the need forrecommendation candidate index to retrieve a copy of the requested pageand determine the users contained therein.

In particular embodiments, web servers 101 transmit a list of servicesavailable for the currently requested page to recommendation engine 112.In such embodiments, services may be specific predetermined algorithmsfor generating candidate objects. For example, the “people you may know”service may filter the recommendable objects down to user nodes withwhom the requesting user shares mutual connections. As another example,the “people you may subscribe to” service may filter the objects to berecommended down to user nodes having public postings that one or moreof the viewing user's friends also subscribe to. In particularembodiments, the “events you may attend” service may limit therecommendable objects to events that the viewing user's friends areattending. In particular embodiments, the “photos of this user” servicemay limit the recommendable objects to photos in which a particular useris tagged. In particular embodiments, the “previous status message”service may limit the recommendable objects to status messages from aparticular user. This disclosure contemplates any suitable number ortype of service. In particular embodiments, web server 101 accesses atable correlating the requested page type (profile page, message thread,event page, etc.) to one or more services, and transmits the list ofservices to recommendation engine 112. In particular embodiments,recommendation engine 112 performs this correlation, and web servers 101merely transmit the page type to recommendation 112. This disclosurecontemplates any suitable method of transmitting a list of services torecommendation candidate index 301.

In particular embodiments, recommendation engine 112 parses therequested page contents for keywords for concept extraction. Forexample, if the user is viewing a message thread, recommendation engine112 may parse the text of the individual messages to filter the objectsto be recommended. For example, if one of the messages discusses“basketball,” recommendation engine may submit the concept “basketball”along with the user ID, to recommendation candidate index 301. Inresponse, recommendation candidate index may return photos by the othermember of the conversation having implicit edges to the concept“basketball.” For example, if the other user owns an album entitled,“Houston Rockets vs. Lakers game,” recommendation candidate index 301may look up the nodes “Houston Rockets” and “Lakers” in the socialgraph, and determine that they both include implicit edges to the largerconcept “basketball”, and present the album, or photos in the album, asa recommended object. In particular embodiments, recommendationcandidate index may utilize computer vision algorithms to determinewhether photos or albums contain implicit edges to concepts. Forexample, content recommendation index 301 may, in the above example,return images of the other user playing basketball. This disclosurecontemplates any suitable means of concept extraction in both therequested page and each individual candidate object.

Regardless of the processing distribution between web server 101,recommendation engine 112, and recommendation candidate index 301,recommendation candidate index 301 eventually returns a list ofcandidate objects for the user ID and page context. Upon receiving thelist of candidates, recommendation candidate index transmits identifiersfor the candidate objects to recommendation content feature provider302, which then returns content features for each of the individualcandidates. Content features may comprise, in particular embodiments,the aggregate global statistics, across all users of social networkingsystem 100, for the content object. In particular embodiments, theaggregate statistics may comprise the click-through rate (CTR). Inparticular embodiments, the click-through rate is the number of timesany user clicks on the content object divided by the number ofimpressions (or displays) to all users. In particular embodiments, theaggregate statistics may be the conversion rate of the content object.In particular embodiments, the “conversion rate” refers to the number oftimes users of social networking system 100 have, upon clicking on asuggested content object and subsequently being taken to the landingpage of the content object, additionally clicked on an element withinthe landing page, divided by the number of impressions of the contentobject. The conversion rate is further discussed with reference to FIG.5. In particular embodiments, the content features include user specificfeatures. For example, when returning the content features for a userobject recommendation, recommendation content feature provider 302 mayquery the social graph to determine the number of mutual friends betweenthe user and the recommended user. In particular embodiments,recommendation engine 112 performs this query. As another example, whenthe candidate object is a photo, recommendation content feature provider302 may return the number of people in the photo. In particularembodiments, recommendation content feature provider 302 may utilizecomputer vision to determine the number of people in a photo.

In particular embodiments, recommendation engine 112 queries userfeature provider 303 with the requesting user's ID immediately uponreceiving the user ID from web servers 101. User feature provider 303may return demographic features about the requesting user by looking upthe user ID's profile page, or querying the social graph with the userID for attribute information associated with the node. In particularembodiments, the user features may include, without limitation, theuser's age, gender, location, religion, political views, interests,number of friends, ethnicity, occupation, and the like. Any informationor attributes that may be determined from user's profile may be a userfeature. In particular embodiments, user feature provider 303 mayprovide an empirical record of the user's historical engagement ofparticular types of content objects. For example, user feature provider303 may return the CTR for a user and “people you may know” userobjects. In particular embodiments, user feature provider 303 may returnthe amount of time that has elapsed since the user has been shown aparticular content object type. For example, the amount of time sincethe last display to a user of “previous status messages.” Thisdisclosure contemplates any suitable set of user features.

In particular embodiments, after obtaining both user and contentfeatures, recommendation engine 112 transmits the features to onlinelearner 304. Online learner 304 returns a model, that is, an array ofvalues of varying weights, that is unique to the set of features forcalculation of an expected value of the user/content object combination.In particular embodiments online learner 304 and recommendation engineutilize logistic regression to calculate the weights and expected value.As commonly known in the art, logistic regression models logisticregression model are discriminitive classifiers for vectors of fixeddimensionality. In this example, the dimensions are the features. Inparticular embodiments, the logistic regression model may includederived features with represent non-linear combinations of other inputfeatures. In particular embodiments, the nonlinearity is achievedthrough multiplication, such that the resulting model linear in thederived features, but will no longer be linear in the original features.Although this disclosure describes a system based on logisticregression, any suitable regression analysis may be used. In particularembodiments, online learner 304 and recommendation engine 112 utilizelinear regression. In particular embodiments, online learner 304 andrecommendation engine 112 utilize nonlinear regression techniques suchas ordinary or weighted least squares. In particular embodiments,Bayesian linear regression is utilized. In particular embodiments,online learner 304 and recommendation engine 112 utilize quantileregression. In particular embodiments, online learner 304 andrecommendation engine 112 utilize nonparametric regression. Thisdisclosure contemplates any suitable method of calculating the expectedvalue of a user clicking or converting a content object given an inputset of user and content features. In particular embodiments, onlinelearner 304 includes a feedback mechanism for updating the model(weights) based on user interaction with rendered content objects. Thisfeedback mechanism is described in further detail with reference to FIG.5.

FIG. 4 illustrates an example method of generating objectrecommendations in response to a page request. At Step 401, a user ofthe social networking system requests a page of content. As previouslydisclosed, the page may be his or her own profile page, another user'sprofile page, a photo album, a photo, or the like.

At Step 402, web server 101 forwards a set of information pertaining tothe request to recommendation engine 112. As previously disclosed, webservers may send, at a minimum, the requester's user ID and the page IDof the requested page to recommendation engine 112. In particularembodiments, web server 101 may transmit the user ID, page ID, and alist of the users contained in the requested page. In particularembodiments, web server 101 may transmit the user ID, page ID, a list ofusers contained in the requested page, and a one or more servicesavailable on the page. In particular embodiments, recommendation engine112 may add, remove, or additionally process any of the data passed toit from web servers 101. In particular embodiments, recommendationengine 112 merely passes data from web server 101. At Step 402,recommendation engine 112 transmits, at a minimum, the user ID and pageID to recommendation candidate index 302. In particular embodiments,recommendation engine 112 transmits the user ID and a page context. Inparticular embodiments, recommendation engine 112 transmits the user ID,page context, and list of users contained in the page. In particularembodiments, recommendation engine 112 transmits the user ID and a listof services available on the page. In particular embodiments,recommendation engine 112 transmits a user ID and a list of the user'sfriends who are contained in the page. As previously stated, thisdisclosure contemplates any suitable amount of processing byrecommendation engine 112. For didactic purposes, however, thisdisclosure assumes recommendation engine merely transmits the user IDand page ID received from web server 101.

At Step 403, recommendation candidate index filters the objects to berecommended by the received user ID and page ID. Recommendationcandidate index filters the candidates based upon the page context;certain pages may only include certain recommendations according to aset of predetermined filtering rules. Additionally, at Step 403,recommendation candidate index filters the set of objects to berecommended by the user ID and the users contained within the requestedpage. For example, for a particular page context only objects of acertain type that are owned by friends of the requesting user may berecommended. After filtering the universe of recommendable objects,recommendation candidate index 301 transmits a list of candidates torecommendation engine 112.

At Step 404, recommendation engine 112 selects the next candidate object(the first object, in this case), and begins the process to calculateexpected values for each individual candidate object received fromrecommendation candidate index 301. In particular embodiments, such asFIG. 4, recommendations for individual candidate objects are calculatediteratively, such as in a “for” or “while” loop. In particularembodiments, the individual candidate IDs are transmitted as largevectors or arrays, and all the expected values are calculated inparallel. This disclosure contemplates any suitable method ofcalculating expected values for a plurality of candidate objects.

At Step 405, upon receiving the list of candidates from recommendationcandidate index 301, recommendation engine 112 queries recommendationcontent feature provider with the candidate ID of the first candidate.Simultaneously, recommendation engine 112 may query user featureprovider 303 with the received user ID. In particular embodiments,recommendation engine 112 may query user feature provider immediatelyupon receiving the user ID from web server 101 (at Step 402).

At Step 406, recommendation engine 112 receives the candidate featuresand the user features from recommendation content feature provider 302and user feature provider 303, respectively. In particular embodiments,recommendation engine 112 may store a set of user features and bypassquerying user feature provider 303 for the remaining candidate objectsin order to reduce network traffic.

At Step 407, recommendation engine 112 queries online learner 304 withthe set of user and candidate features. In response, online learner 304returns an array of weights, the weights having different valuesdepending on the set of input features. As previously disclosed, onlinelearner 304 may utilize a logistic regression to calculate the arrayweights returned to recommendation engine 112.

At Step 408, recommendation engine 112 calculates an expected value forthe candidate object. In particular embodiments, the expected value maybe represented as C1*CTR+C2*Conversion_rate, where the constants arepredetermined values. This disclosure contemplates any suitable methodof calculating an expected value for a particular user and contentobject. At Step 409, recommendation engine determines whether it hascalculated expected values for all the candidate objects returned fromrecommendation candidate index 301 in Step 403. If so, the processcontinues to Step 410; if not, the process repeats for the nextcandidate object in the queue.

At Step 410, recommendation engine 112 ranks all the candidate objects,and renders a predetermined number of the candidate objects having thehighest expected value to the user. In particular embodiments, thehighest ranked candidate objects may be displayed on the right handcolumn of a profile page. In particular embodiments, they may bedisplayed on another area of the user display. In particularembodiments, recommendation engine 112 may be an application thatcommunicates via an application user interface of a social plug-in thatresides on a third-party website.

FIG. 5 illustrates an example to update weights of the model in onlinelearner 304 in response to user clicks or conversions. Because of thelarge number of impressions and user interactions with recommendedobjects in social networking system 100, it is imperative that the modelgenerated by online learner 304 stay up to date in substantially realtime in order to reflect habits or preferences of users of the socialnetworking system.

At Step 500, a particular content object is rendered on the display of aparticular user. Online learner 304 begins logging the user's activityand interactions with the rendered content object for a predeterminedamount of time. In particular embodiments, a timeout timer is startedupon rendering the content object to the user, also called “registeringan impression.”

At Step 501, online learner 304 waits for the user to click on theparticular content object. If no click is received and the timeout timerexpires at the end of the predetermined interval, the process assumesthe particular content object was not interesting to the user, andregisters the impression as a negative example in Step 510. Inparticular embodiments, the predetermined interval is 15 minutes. thisdisclosure contemplates a timeout period of any arbitrary duration.

Similarly, in particular embodiments at Step 501, if the users clicks an“X” icon in the content object, indicating that the user no longer wantsto see the content object or similar content objects, online learner 304may register this as a highly negative example, and proceed to Step 502.In particular embodiments, hiding, closing, or “x-ing out” a contentobject is weighted as a more negative example than merely notinteracting with the content object. In particular embodiments, thissignal does not affect the model weights. This disclosure contemplatesany suitable weighting of a “hide” command relative to a timeout withoutclicking the content object.

At Step 501, if the user clicks on the content object but does not“x-out” the content object, online learner 304 considers the interactiona “positive click” and increments the CTR of the content object at Step503. Additionally, after a user clicks on a recommended content object,web server 101 renders the landing page of the object. The landing pagefor a given object may vary depending on the type of object. Forexample, when a user clicks another user who is a friend, the landingpage is the other user's profile. As another example, when a user clicksanother user who is not a friend, the landing page is the other user'spublic search page. As another example, when a user clicks an eventobject, the landing page is the event page. This disclosure contemplatesany suitable landing page for any suitable content object.

At Step 505, online learner 505 determines whether the user converted onthe object. For the purposes of this application, “converting” upon acontent object means clicking on an element of its landing page. Thecontent of each landing page is also different for each content object.For example, where a landing page is a user profile who is not friendswith the viewing user, the selectable elements may be, in particularembodiments, adding the user as a friend, messaging the user, and“poking” the user. As another example, if the landing page is an eventpage, the selectable elements may be to RSVP to the event, invite otherguests to the event, or view the guestlist. This disclosure contemplatesany suitable user-selectable elements in the landing pages of anysuitable content object.

If the user converts on the content object by clicking an element of inthe landing page of the content object, online learner 304 proceeds toStep 506. At Step 506, the weight of the positive example is altereddepending on the type of conversion type. For example, if the landingpage is the public search profile of a non-friend user, online learner304 may consider sending a friend request to be a strong positiveexample, sending a message to be a slightly less strong positiveexample, and poking the user as merely a strong positive example (thoughhigher than merely clicking on the content object). As another example,RSVPing to an event may be a strong positive conversion, whereas writingon the wall of the event may be a slightly weaker positive conversion,and viewing the guestlist may be considered a weak positive conversion.In particular embodiments, the relative strength of the variousconversions and the assignment of weights to the different actions arepredetermined by social networking system 100. This disclosurecontemplates any suitable weight assignment.

If the user clicks on the content object at Step 501, but does notfurther convert on the content object within the timeout period, onlinelearner at Step 508 rates the interaction as a positive example. Inparticular embodiments, the strength of the positive example is lowerthan any user interactions that involve the initial click and aconversion as described above.

At Step 509, online learner updates the weights in the model. Aspreviously described, the model weights may be obtained through logisticregression, and the positive or negative example may fed into the modelas another data point. Because online learner 304 is aware of the userfeatures for the user interacting with the content object, as well asthe content features, the data point fed into online learner 304 canaccurately encapsulate any correlative or casual relationships betweenthe user and the content object. As previously disclosed, thisdisclosure contemplates any suitable regression analysis.

While the foregoing embodiments may be implemented in a variety ofnetwork configurations, the following illustrates an example networkenvironment for didactic, and not limiting, purposes. FIG. 6 illustratesan example network environment 600. Network environment 600 includes anetwork 610 coupling one or more servers 620 and one or more clients 630to each other. Network environment 600 also includes one or more datastorage 640 linked to one or more servers 620. Particular embodimentsmay be implemented in network environment 600. For example, socialnetworking system frontend 120 may be written in software programshosted by one or more servers 620. For example, event database 102 maybe stored in one or more storage 640. In particular embodiments, network610 is an intranet, an extranet, a virtual private network (VPN), alocal area network (LAN), a wireless LAN (WLAN), a wide area network(WAN), a metropolitan area network (MAN), a portion of the Internet, oranother network 610 or a combination of two or more such networks 610.This disclosure contemplates any suitable network 610.

One or more links 650 couple a server 620 or a client 630 to network610. In particular embodiments, one or more links 650 each includes oneor more wired, wireless, or optical links 650. In particularembodiments, one or more links 650 each includes an intranet, anextranet, a VPN, a LAN, a WLAN, a WAN, a MAN, a portion of the Internet,or another link 650 or a combination of two or more such links 650. Thisdisclosure contemplates any suitable links 650 coupling servers 620 andclients 630 to network 610.

In particular embodiments, each server 620 may be a unitary server ormay be a distributed server spanning multiple computers or multipledatacenters. Servers 620 may be of various types, such as, for exampleand without limitation, web server, news server, mail server, messageserver, advertising server, file server, application server, exchangeserver, database server, or proxy server. In particular embodiments,each server 620 may include hardware, software, or embedded logiccomponents or a combination of two or more such components for carryingout the appropriate functionalities implemented or supported by server620. For example, a web server is generally capable of hosting websitescontaining web pages or particular elements of web pages. Morespecifically, a web server may host HTML files or other file types, ormay dynamically create or constitute files upon a request, andcommunicate them to clients 630 in response to HTTP or other requestsfrom clients 630. A mail server is generally capable of providingelectronic mail services to various clients 630. A database server isgenerally capable of providing an interface for managing data stored inone or more data stores.

In particular embodiments, one or more data storages 640 may becommunicatively linked to one or more servers 620 via one or more links650. In particular embodiments, data storages 640 may be used to storevarious types of information. In particular embodiments, the informationstored in data storages 640 may be organized according to specific datastructures. In particular embodiment, each data storage 640 may be arelational database. Particular embodiments may provide interfaces thatenable servers 620 or clients 630 to manage, e.g., retrieve, modify,add, or delete, the information stored in data storage 640.

In particular embodiments, each client 630 may be an electronic deviceincluding hardware, software, or embedded logic components or acombination of two or more such components and capable of carrying outthe appropriate functions implemented or supported by client 630. Forexample and without limitation, a client 630 may be a desktop computersystem, a notebook computer system, a netbook computer system, ahandheld electronic device, or a mobile telephone. This disclosurecontemplates any suitable clients 630. A client 630 may enable a networkuser at client 630 to access network 630. A client 630 may enable itsuser to communicate with other users at other clients 630.

A client 630 may have a web browser 632, such as MICROSOFT INTERNETEXPLORER, GOOGLE CHROME or MOZILLA FIREFOX, and may have one or moreadd-ons, plug-ins, or other extensions, such as TOOLBAR or YAHOOTOOLBAR. A user at client 630 may enter a Uniform Resource Locator (URL)or other address directing the web browser 632 to a server 620, and theweb browser 632 may generate a Hyper Text Transfer Protocol (HTTP)request and communicate the HTTP request to server 620. Server 620 mayaccept the HTTP request and communicate to client 630 one or more HyperText Markup Language (HTML) files responsive to the HTTP request. Client630 may render a web page based on the HTML files from server 620 forpresentation to the user. This disclosure contemplates any suitable webpage files. As an example and not by way of limitation, web pages mayrender from HTML files, Extensible Hyper Text Markup Language (XHTML)files, or Extensible Markup Language (XML) files, according toparticular needs. Such pages may also execute scripts such as, forexample and without limitation, those written in JAVASCRIPT, JAVA,MICROSOFT SILVERLIGHT, combinations of markup language and scripts suchas AJAX (Asynchronous JAVASCRIPT and XML), and the like. Herein,reference to a web page encompasses one or more corresponding web pagefiles (which a browser may use to render the web page) and vice versa,where appropriate.

FIG. 7 illustrates an example computing system architecture, which maybe used to implement a server. In one embodiment, hardware system 700comprises a processor 702, a cache memory 704, and one or moreexecutable modules and drivers, stored on a tangible computer readablemedium, directed to the functions described herein. Additionally,hardware system 700 includes a high performance input/output (I/O) bus706 and a standard I/O bus 708. A host bridge 708 couples processor 702to high performance I/O bus 706, whereas I/O bus bridge 712 couples thetwo buses 706 and 708 to each other. A system memory 714 and one or morenetwork/communication interfaces 716 couple to bus 706. Hardware system700 may further include video memory (not shown) and a display devicecoupled to the video memory. Mass storage 717, and I/O ports 720 coupleto bus 708. Hardware system 700 may optionally include a keyboard andpointing device, and a display device (not shown) coupled to bus 708.Collectively, these elements are intended to represent a broad categoryof computer hardware systems, including but not limited to generalpurpose computer systems based on the x76-compatible processorsmanufactured by Intel Corporation of Santa Clara, Calif., and thex76-compatible processors manufactured by Advanced Micro Devices (AMD),Inc., of Sunnyvale, California, as well as any other suitable processor.

The elements of hardware system 700 are described in greater detailbelow. In particular, network interface 716 provides communicationbetween hardware system 700 and any of a wide range of networks, such asan Ethernet (e.g., IEEE 702.3) network, a backplane, etc. Mass storage717 provides permanent storage for the data and programming instructionsto perform the above-described functions implemented in the servers 22a, 22 b, whereas system memory 714 (e.g., DRAM) provides temporarystorage for the data and programming instructions when executed byprocessor 702. I/O ports 620 are one or more serial and/or parallelcommunication ports that provide communication between additionalperipheral devices, which may be coupled to hardware system 700.

Hardware system 700 may include a variety of system architectures; andvarious components of hardware system 700 may be rearranged. Forexample, cache 704 may be on-chip with processor 702. Alternatively,cache 704 and processor 702 may be packed together as a “processormodule,” with processor 702 being referred to as the “processor core.”Furthermore, certain embodiments may not require nor include all of theabove components. For example, the peripheral devices shown coupled tostandard I/O bus 708 may couple to high performance I/O bus 706. Inaddition, in some embodiments, only a single bus may exist, with thecomponents of hardware system 700 being coupled to the single bus.Furthermore, hardware system 700 may include additional components, suchas additional processors, storage devices, or memories.

In one implementation, the operations of the embodiments describedherein are implemented as a series of executable modules run by hardwaresystem 700, individually or collectively in a distributed computingenvironment. In a particular embodiment, a set of software modulesand/or drivers implements a network communications protocol stack,browsing and other computing functions, optimization processes, and thelike. The foregoing functional modules may be realized by hardware,executable modules stored on a computer readable medium, or acombination of both. For example, the functional modules may comprise aplurality or series of instructions to be executed by a processor in ahardware system, such as processor 702. Initially, the series ofinstructions may be stored on a storage device, such as mass storage717. However, the series of instructions can be tangibly stored on anysuitable storage medium, such as a diskette, CD-ROM, ROM, EEPROM, etc.Furthermore, the series of instructions need not be stored locally, andcould be received from a remote storage device, such as a server on anetwork, via network/communications interface 716. The instructions arecopied from the storage device, such as mass storage 717, into memory714 and then accessed and executed by processor 702.

An operating system manages and controls the operation of hardwaresystem 700, including the input and output of data to and from softwareapplications (not shown). The operating system provides an interfacebetween the software applications being executed on the system and thehardware components of the system. Any suitable operating system may beused, such as the LINUX Operating System, the Apple Macintosh OperatingSystem, available from Apple Computer Inc. of Cupertino, Calif., UNIXoperating systems, Microsoft (r) Windows(r) operating systems, BSDoperating systems, and the like. Of course, other implementations arepossible. For example, the nickname generating functions describedherein may be implemented in firmware or on an application specificintegrated circuit.

Furthermore, the above-described elements and operations can becomprised of instructions that are stored on storage media. Theinstructions can be retrieved and executed by a processing system. Someexamples of instructions are software, program code, and firmware. Someexamples of storage media are memory devices, tape, disks, integratedcircuits, and servers. The instructions are operational when executed bythe processing system to direct the processing system to operate inaccord with the disclosure. The term “processing system” refers to asingle processing device or a group of inter-operational processingdevices. Some examples of processing devices are integrated circuits andlogic circuitry. Those skilled in the art are familiar withinstructions, computers, and storage media.

Herein, “or” is inclusive and not exclusive, unless expressly indicatedotherwise or indicated otherwise by context. Therefore, herein, “A or B”means “A, B, or both,” unless expressly indicated otherwise or indicatedotherwise by context. Moreover, “and” is both joint and several, unlessexpressly indicated otherwise or indicated otherwise by context.Therefore, herein, “A and B” means “A and B, jointly or severally,”unless expressly indicated otherwise or indicated otherwise by context.

Herein, reference to a computer-readable non-transitory storage mediummay include a semiconductor-based or other integrated circuit (IC) (suchas, for example, a field-programmable gate array (FPGA) or anapplication-specific IC (ASIC)), a hard disk, an HDD, a hybrid harddrive (HHD), an optical disc, an optical disc drive (ODD), amagneto-optical disc, a magneto-optical drive, a floppy disk, a floppydisk drive (FDD), magnetic tape, a holographic storage medium, asolid-state drive (SSD), a RAM-drive, a SECURE DIGITAL card, a SECUREDIGITAL drive, another suitable computer-readable non-transitory storagemedium, or a suitable combination of these, where appropriate. Acomputer-readable non-transitory storage medium may be volatile,non-volatile, or a combination of volatile and non-volatile, whereappropriate.

This disclosure encompasses all changes, substitutions, variations,alterations, and modifications to the example embodiments herein that aperson having ordinary skill in the art would comprehend. By way ofexample, while embodiments have been described as operating inconnection with a social networking website, the disclosure can be usedin connection with any communications facility that supports webapplications. Furthermore, in some embodiments the term “web service”and “web-site” may be used interchangeably and additionally may refer toa custom or generalized API on a device, such as a mobile device (e.g.,cellular phone, smart phone, personal GPS, personal digital assistance,personal gaming device, etc.), that makes API calls directly to aserver. Moreover, reference in the appended claims to an apparatus orsystem or a component of an apparatus or system being adapted to,arranged to, capable of, configured to, enabled to, operable to, oroperative to perform a particular function encompasses that apparatus,system, component, whether or not it or that particular function isactivated, turned on, or unlocked, as long as that apparatus, system, orcomponent is so adapted, arranged, capable, configured, enabled,operable, or operative.

1. A method comprising: by one or more computing devices, receiving arequest from a first user for a content page; by one or more computingdevices, in response to the request, identifying at least one contentpage, wherein the content page is associated with a page identifier; byone or more computing devices, identifying a plurality of content itemsbased at least in part on a plurality of content features associatedwith the content page; by one or more computing devices, ranking theplurality of content items based at least in part on a plurality of userfeatures associated with the first user; and by one or more computingdevices, providing to the first user, with the requested content page,one or more of the plurality of content items as recommendations to thefirst user based on the ranking of the content items.
 2. The method ofclaim 1, wherein the content page is associated with a content context,the content context comprising a content page identifier.
 3. The methodof claim 1, wherein the content page is associated with a contentcontext, the content context comprising a list of one or more otherusers in the requested content.
 4. The method of claim 1, wherein thecontent page is associated with a content context, the content contextcomprising a list of one or more services for the content.
 5. The methodof claim 1, wherein the plurality of user features comprise the firstuser's age.
 6. The method of claim 1, wherein the plurality of userfeatures comprise the first user's gender.
 7. The method of claim 1,wherein the plurality of user features comprise the first user's lastrequest of the content.
 8. The method of claim 1, wherein the contentfeatures comprise aggregate statistics for the content page.
 9. Themethod of claim 8, wherein the aggregate statistics comprise the clickthrough rate for the content page.
 10. The method of claim 8, whereinthe aggregate statistics comprise the conversion rate for one or moreobjects associated with the content page.
 11. The method of claim 8,wherein the aggregate statistics comprise the number of impressions in apredetermined period of time for one or more objects associated with thecontent page.
 12. The method of claim 1, wherein the ranking isperformed by a recommendation model, the recommendation model comprisinga set of weights for each of the user and content features calculatedvia one or more statistical models.
 13. The method of claim 12, whereinthe one or more statistical models comprises logistic regression. 14.The method of claim 1, further comprising: receiving an examplecomprising a positive signal or a negative signal for a particular setof user features and content features; and updating the recommendationmodel based on the example.
 15. The method of claim 14, wherein: theexample is positive if a viewing user having the particular set of userfeatures fails to convert on a viewed content item having the particularset of content features after an impression, and the example is negativeif the viewing user having the particular set of user features convertson the viewed content item having the particular set of content featuresafter the impression.
 16. The method of claim 15, wherein converting onthe viewed object comprises clicking on a link displayed in a landingpage of the viewed object.
 17. A non-transitory, computer-readable mediacomprising instructions operable, when executed by one or more computingsystems, to: receive a request from a first user for a content page; inresponse to the request, identify at least one content page, wherein thecontent page is associated with a page identifier; identify a pluralityof content items based at least in part on a plurality of contentfeatures associated with the content page; rank the plurality of contentitems based at least in part on a plurality of user features associatedwith the first user; and provide to the first user, with the requestedcontent page, one or more of the plurality of content items asrecommendations to the first user based on the ranking of the contentitems.
 18. The media of claim 17, wherein the content page is associatedwith a content context, the content context comprising a content pageidentifier.
 19. The media of claim 17, the instructions furtheroperable, when executed by one or more computing systems, to: receive anexample comprising a positive signal or a negative signal for aparticular set of user features and content features; and update therecommendation model based on the example.
 20. A system comprising: oneor more processors; and one or more computer-readable non-transitorystorage media coupled to one or more of the processors and comprisinginstructions operable when executed by one or more of the processors tocause the system to: receive a request from a first user for a contentpage; in response to the request, identify at least one content page,wherein the content page is associated with a page identifier; identifya plurality of content items based at least in part on a plurality ofcontent features associated with the content page; rank the plurality ofcontent items based at least in part on a plurality of user featuresassociated with the first user; and provide to the first user, with therequested content page, one or more of the plurality of content items asrecommendations to the first user based on the ranking of the contentitems.